Variations of Logistic Regression with Stochastic Gradient Descent
نویسنده
چکیده
In this paper, we extend the traditional logistic regression model(LR) to the bounded logistic regression model(BLR) and compare them. We also derive the update rules of both model using stochastic gradient desent(SGD). The effects of choosing different learning rate schedule, stopping conditions, parameters initialization and learning algorithm settings are also discussed. We get the accuracy rate of 83.80% for LR model and 84.75% for BLR model on the test set.
منابع مشابه
Comparison of Modern Stochastic Optimization Algorithms
Gradient-based optimization methods are popular in machine learning applications. In large-scale problems, stochastic methods are preferred due to their good scaling properties. In this project, we compare the performance of four gradient-based methods; gradient descent, stochastic gradient descent, semi-stochastic gradient descent and stochastic average gradient. We consider logistic regressio...
متن کاملLazy Sparse Stochastic Gradient Descent for Regularized Mutlinomial Logistic Regression
Stochastic gradient descent efficiently estimates maximum likelihood logistic regression coefficients from sparse input data. Regularization with respect to a prior coefficient distribution destroys the sparsity of the gradient evaluated at a single example. Sparsity is restored by lazily shrinking a coefficient along the cumulative gradient of the prior just before the coefficient is needed. 1...
متن کاملADAPTIVITY OF AVERAGED STOCHASTIC GRADIENT DESCENT Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
In this paper, we consider supervised learning problems such as logistic regression and study the stochastic gradient method with averaging, in the usual stochastic approximation setting where observations are used only once. We show that after N iterations, with a constant step-size proportional to 1/R √ N where N is the number of observations and R is the maximum norm of the observations, the...
متن کاملDoubly Stochastic Variational Bayes for non-Conjugate Inference
We propose a simple and effective variational inference algorithm based on stochastic optimisation that can be widely applied for Bayesian non-conjugate inference in continuous parameter spaces. This algorithm is based on stochastic approximation and allows for efficient use of gradient information from the model joint density. We demonstrate these properties using illustrative examples as well...
متن کاملAdaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau...
متن کامل